Method of defining and performing dynamic user-computer interaction, computer guided navigation, and application integration for any procedure, instructions, instructional manual, or fillable form

ABSTRACT

Various embodiments described in this disclosure relate to methods and computer-based systems that implement those methods for overlaying or superimposing computer-user interaction widgets and application interface widgets on top of content blocks contained within digitized documents which represent written procedures or instructions, instructional manuals and fillable forms, in order to support means of providing dynamic user-computer interaction and computer guided navigation, as well as application functions and data integration, driven through structured interaction and integration metadata definitions, as it relates to said content blocks, during job performance.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of and priority to U.S. ProvisionalPatent Application Ser. No. 62/676,197 filed on May 24, 2018 andentitled “Method of Defining and Performing Dynamic Human-ComputerInteraction, Computer Guided Navigation, And Application Integration forAny Procedure, Instructions, Instructional Manual, or Fillable Form,”which application is expressly incorporated herein by reference in itsentirety.

BACKGROUND Background and Relevant Art

Across any industry that requires a structured approach to jobperformance, many jobs performed by people are guided by writtenprocedures and work instructions, which ensures reliable and consistentperformance of tasks and minimizes the risk of user errors. Eveninstructional manuals for goods, requiring setup and maintenance,contain instructions that need to be followed by a person using suchgoods.

Both instructional manuals as well as written procedures/instructionsdirect the actions of a person performing the job in a defined sequence,while, optionally, providing other information and documents such asdiagrams, drawings, illustrations, images, and fillable forms.

Instructional manuals, written procedures and work instructions areoften presented to one who performs a job in the form of paper documentsor an equivalent digitized document, such as a PDF, on a computerdevice.

In more mission-critical industries, such as Nuclear or Oil & Gas, fieldworkers often carry a large stack of paper work packages, or anequivalent digitized set of documents on a computer device, to the jobperformance site just to complete a single work order. Often suchprocedures and instructions encompass multiple different scenarios andthus require complex navigation of documents, whether physical documentsor electronic, to the applicable scenario by the field worker.

Field workers must often manually sift through volumes of informationpertaining to irrelevant scenarios in order to navigate to theapplicable steps, diagrams and instructions. Not only does this reducethe “wrench time”, or the time spent actually performing the task onhand, but also may lead to unintentional user errors. Paper proceduresand instructions or their equivalent digitized documents providedrequire manual and time consuming navigation to relevant information andthe next step, computational support, placekeeping of instruction steps,verification of tools and equipment pertaining to theprocedure/instructions, verified acknowledgment of Cautions and Noteswithin instructions or other content blocks, verified completion ofsignatures, and support for other aspects of job performance.

Furthermore, non-interactive written procedures/instructions only allowfor job monitoring once the job is complete, and only based on anemployee's report of the job.

There are various innovative approaches in defining and performinguser-computer interaction and computer guided navigation when it comesto the performance of written procedures/instructions. However, anyapproach that requires fundamentally changing the wayprocedure/instructions are authored will suffer from significant changemanagement costs and user adoption issues. For example, a typicalnuclear utility invests tens of millions of dollars in the process ofwriting, reviewing, approving, and organizing tens of thousands of workinstructions and procedures. In a nuclear utility, procedure writerscurrently use advanced, mature and full-featured text authoring toolssuch as Microsoft Word or Adobe FrameMaker to define procedures andwork-instructions. Abruptly changing that with a new tool for authoringinstructions as well as user-computer interaction is not only a colossalundertaking but also suffers from tremendous change management costs inusing the new process and tools for procedure-writing. Besides thechange management costs, such approaches pose a significant risk ofadoption and user acceptance as currently procedure writers areaccustomed to mature software for authoring instruction (without theinteraction or application integration).

The subject matter claimed herein is not limited to embodiments thatsolve any disadvantages or that operate only in environments such asthose described above. Rather, this background is only provided toillustrate one exemplary technology area where some embodimentsdescribed herein may be practiced.

BRIEF SUMMARY

The scope of the present invention is not limited to any degree by thestatements within this summary.

Various embodiments of the disclosure relate, generally, to computerimplemented systems and methods to define user-computer interactions,computer guided navigation and application integration for digitizedwritten procedures/instructions, instructional manuals and/or fillableforms (hereon referred to collectively or individually as the digitizeddocument), in a structured metadata format containing the interactionand interface definitions together with the corresponding content blocksof the digitized document, and to perform the defined interactions andintegrations guided through the structured metadata at job performancetime. The computer guided interaction and application interfacedefinitions may be defined through an interaction authoring tool(Interaction Authoring Tool—IAT) by overlaying or superimposinguser-computer interaction widgets (heron referred to as interactionwidgets) and application integration widgets (hereon referred to asinterface widgets) over content blocks of the digitized document toencapsulate each content block through a corresponding widget and thenconfiguring the interaction and interface properties of said widgetincluding configuring its sequence of appearance statically orconditionally based on dynamic data, relative to other widgets, forcomputer-aided navigation at job performance time. The computer-guidedinteraction and application interface definitions (Interaction andInterface Definition Metadata—IIDM), together with content blocksencapsulated through the widgets, are stored in a persistent computerdata store as structured XML, or functionally and sufficientlyequivalent data structures. Based on some embodiments of the disclosurethe IIDM may also be generated programmatically. At job performancetime, computer guided interactions and application integration may beperformed within an Interaction Performance Tool (IPT) driven by theinterpretation of the IIDM.

Other embodiments of the disclosure relate, generally, to definingconfiguration driven interaction widgets which encapsulate contentblocks from the digitized document with each widget supporting specifictypes of user-computer interactions, as it relates to its correspondingcontent block, and embedding select widgets within other widgets formore granular encapsulation and context sensitive interactions.

Yet other embodiments of the disclosure relate, generally, to methodsfor generating IIDM programmatically or through IAT, capturing andaggregating information about the dynamic user-computer interactionsduring job performance to provide job performance monitoring, statisticsand analytics and supporting computer-guided navigation from thecontents encapsulated within one widget to contents encapsulated withinanother through static sequencing of widgets or dynamically through theautomatic evaluation of conditional statements based on dynamic data.Furthermore, to methods accommodating advanced computer guidance for jobrequirements such as repeated action steps and other advancedinteractions through different types of superimposed/overlaid and linkedinteraction widgets.

Yet, other embodiments of the disclosure relate, generally, to methodsof defining, and supporting at job performance time, the bidirectionalintegration of data and automatic dispatch of application functions fromother applications to/from content blocks of an existing digital writtenprocedure/instructions or fillable form, integrations which may bedefined through the selection of software service interfaces with typedinput/outputs such as standards-based Web Services (e.g., WSDL/SOAP orRESTful) that are automatically associated with a set of interfacewidgets that may be placed inside of any interaction or any widgetencapsulating said content blocks (container widget). Furthermore, someembodiments of the disclosure relate, generally, to providing integratedmethods through typed data interface widgets, to gather user inputsduring job performance as it relates to the contents of one interactionwidget and making such data referenceable by and available to otherproceeding widgets to support dynamic computer-aided navigation andreduce user errors during job performance.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used as an aid in determining the scope of the claimed subjectmatter.

Additional features and advantages will be set forth in the descriptionwhich follows, and in part will be obvious from the description, or maybe learned by the practice of the teachings herein. Features andadvantages of the invention may be realized and obtained by means of theinstruments and combinations particularly pointed out in the appendedclaims. Features of the present invention will become more fullyapparent from the following description and appended claims, or may belearned by the practice of the invention as set forth hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

The above, and other aspects, features, and advantages of severalembodiments of the present disclosure will be more apparent to one ofordinary skill in the art from the following Detailed Description aspresented in conjunction with the following several appended drawings,understanding that these drawings depict only typical embodiments andare not therefore to be considered to be limiting in scope, including:

FIG. 1 shows a flowchart of an embodiment of a method to defineuser-computer interaction and computer-aided navigation throughsuperimpose/overlay of interaction widgets on top of content blocks ofany digitized written procedure/instructions, instructional manual andfillable forms, resulting in structured metadata definitions.

FIG. 2 shows a flowchart of an embodiment of a method to defineapplication and data integration through superimpose/overlay ofinterface widgets on top of content blocks, or portions thereof, of anydigitized written procedure/instructions, instructional manual andfillable forms, resulting in structured metadata definitions.

FIG. 3 shows a flowchart of a method to provide/perform, at jobperformance time, user-computer interaction and computer-aidednavigation as well as application and data integration throughsuperimpose/overlay of interaction and interface widgets on top ofcontent blocks of any digitized written procedure/instructions andfillable forms based on structured metadata definitions.

FIG. 4 shows an example of an embodiment of the Interaction AuthoringTool (IAT), displaying an example of the digitized document, and anexample of an interaction widget palette, with a Section interactionwidget, selected from the palette and, superimposed on the content blockof the first section of the digitized document together with an exampleof some of the widget's properties and attributes.

FIG. 5 shows the same digitized document as FIG. 4, with moresuperimposed and linked interaction widgets, where one of theinteraction widgets contains child interaction widgets (or interactionsub-widgets) that correspond to instruction steps within the contentblocks contained by the parent widget and with more space being added,dynamically, to the displayed digitized document to accommodate spacefor the Acknowledgment related interactions of some superimposedinteraction widgets.

FIG. 6 shows the use of a Conditional Action step widget, within IAT, todefine automatic branching and navigation to the next interaction widgetbased on dynamic user input later provided at job performance timewithin the Interaction Performance Tool (IPT).

FIG. 7 shows an example of an embodiment of the Interaction PerformanceTool (IPT), performing user-computer interaction, computer-guidednavigation and placekeeping based on the superimposed/overlaidinteraction widgets and their configured attributes as partiallyshown/defined in FIG. 6.

Elements in the several figures are illustrated for simplicity andclarity and have not necessarily been drawn to scale. Also, common, butwell-understood elements that are useful or necessary in commerciallyfeasible embodiments are often not depicted in order to facilitate aless obstructed view of these various embodiments of the presentdisclosure.

DETAILED DESCRIPTION

The efficiency of performing procedures/instructions may besignificantly improved when dynamic information about the job, workenvironment or equipment/tools is presented, in a graphical userinterface, to the person performing the job. Indeed, embodiments areable to provide information to a user that was not previously available.Some of this information resides in the core backend enterpriseapplications. Also, data entered during job performance, that may becaptured using fillable forms or segments thereof, most often belong tothe core backend enterprise applications such as Asset Management orWork Management systems. Furthermore, sometimes at job performance time,the ability to conveniently access a function of those backendapplications while performing procedures/instructions, for example beingable to create a work request through a Work Management system, maysignificantly improve job efficiency. Thus, embodiments implement animproved computing system that is able to display a standardizedprocedure/instruction manual or fillable form, while also providinginteractive access with additional enterprise data while the user isinteracting with the manual or form. Existing writtenprocedures/instructions, instructional manuals or fillable forms,whether in paper form or as a digitized document do not provide suchfunctionality. For example, existing computer systems providing suchmanuals and forms do not also include interactive instructions guiding auser through the form, nor do such systems provide access to additionalenterprise data related to the form.

By separating concerns as it relates to instruction authoring vsinteraction/interface authoring, applying a method of embodiments ofthis invention minimizes change management costs and results in anon-intrusive and incremental approach to adding computer guidedinteraction and application integration to the performance of existing,as well as new, written procedures/instructions without requiring anysignificant change to the established existing processes of anorganization and, instead, by adding a layer of processes, and computerguided interaction, on top of what already exists.

The application of a method of embodiments of this invention provides anon-intrusive, practical and incremental approach and means to improvework efficiency and reduce user error through computer guidedinteraction for navigation to the relevant job scenario, information andsteps, computational support, automatic placekeeping of instructionsteps, verification of tools and equipment pertaining theprocedure/instructions, verified acknowledgment of Cautions and Noteswithin procedures, and for other aspects of performing a job.

The following description is meant to enable one of ordinary skill inthe art to practice the disclosed embodiments by the way of example andspecific details of its key elements. The use of specific details,arrangement of components, specified features, examples or the like donot signify a preferred embodiment and are not intended to limit thescope of this invention to said details or examples. In addition, asubset of the components of this embodiment may be arranged by one ofordinary skill in the art in different configurations than specified andall drawings accompanying this disclosure are for illustrative purposesonly. Therefore, one skilled in the relevant art will recognize,however, that the embodiments of the present disclosure can be practicedwithout one or more of the specific details, or with conventionalmethods used in the industry or other methods, components, and so forth.

In general, well-known structures and operations are not shown ordescribed in detail to avoid obscuring aspects of the presentdisclosure. Other elements not described herein relate to variousconventional methods, components and acts that one with ordinary skillin the art may incorporate in accord with the embodiment of thisdisclosure. For example, details concerning exception-handling, timingand other similar considerations have been omitted where such detail isnot necessary for obtaining a complete understanding of this closure byone of ordinary skill in the art. Furthermore, any alterations andfurther modifications in the illustrated methods, and such furtherapplication of the principles of the invention as illustrated herein arecontemplated as would normally occur with one skilled in the artincorporating the method of this invention.

As noted above, the following description is not to be taken in alimiting sense, but is made merely for the purpose of describing thegeneral principles of exemplary embodiments. Many additional embodimentsof this invention are possible. It is understood that no limitation ofthe scope of the invention is thereby intended. Reference throughoutthis specification to “one embodiment,” “an embodiment,” or similarlanguage means that a particular feature, structure, or characteristicthat is described in connection with the embodiment is included in atleast one embodiment of the present disclosure. Thus, appearances of thephrases “in one embodiment,” “in an embodiment,” and similar languagethroughout this specification may, but do not necessarily, all refer tothe same embodiment. Also, the phrases “at least one,” “one or more,”and “and/or” are open-ended expressions that are both conjunctive anddisjunctive in operation.

Moreover, no requirement exists for a system or method to address eachand every problem sought to be resolved by the present disclosure, forsuch to be encompassed by the present claims. Furthermore, no element,component, or method step in the present disclosure is intended to bededicated to the public regardless of whether the element, component, ormethod step is explicitly recited in the claims. However, that variouschanges and modifications in form, composition of components and soforth may be made, without departing from the spirit and scope of thepresent disclosure, as set forth in the appended claims, as may beapparent to those of ordinary skill in the art, are also encompassed bythe present disclosure.

In one embodiment of the disclosure, generally, a digitized form of thewritten procedure/instructions, instructional manual and/or fillableforms, in PDF with a rich content format or any other electronic format,referred to as the digitized document, is imported into an InteractionAuthoring Tool (IAT) as illustrated at step 101 of FIG. 1 and displayedas illustrated at step 102.1 in the background using a scrolledgraphical user interface window on a display of a computing device. TheIAT 402 also displays, in a separate window, a rich palette ofconfigurable interaction widgets as illustrated at step 102.2 where eachwidget in the palette, generally or specifically, corresponds to adifferent content type within a typical digitized document. FIG. 1 showsa flowchart that depicts the simplified steps, in one embodiment of thisdisclosure, used to define user-computer interaction and computer-aidednavigation through superimpose/overlay of interaction widgets over thecontent blocks of the digitized document. In FIG. 1, steps 104 and 105,the person defining the interactions (the interaction designer)superimposes (to show the content underneath) or overlays (to block thecontent underneath or portions thereof) a selected widget, from saidpalette, over (or on top of) a corresponding content block and resizesthe widget to contain and snag the desired content block (depicted inFIG. 1, step 106).

For example, the interaction designer, may superimpose a Cautioninteraction widget, over the Caution content block of the underlyingprocedure/instructions, resize the widget to contain the written Cautionsection, then set the attributes of the widget such as job performancesequence designator (e.g., sequence number or other ordered designator),that this instance of Caution requires acknowledgment, where to placeand what to label (with a label for pre-acknowledgment and another forpost acknowledgment) the means of acknowledgment inside the widget, thedisplay properties of the acknowledgment, so on so forth, to indicate anacknowledgement of the Caution section is required before the user maymove to the next content block of the procedure/instructions at jobperformance time, and to set other properties available on the widgetthat may include look and feel properties. Similarly, the interactiondesigner may superimpose Step interaction widgets over each step of theprocedure/instructions, set their display attributes, and sequence,through e.g., sequence attribute numbers, and link the steps widgetsbased on the order of performance, or conditionally, through binaryquestions, or more complex conditional statements, based on differentjob scenarios and thus provide automatic, dynamic, computer guidednavigation of instructions within the Interaction Performance Tool (IPT)702 (see FIG. 7) at job performance time.

FIG. 4 shows one embodiment of IAT 402, where an example of writtenprocedure/instructions, used for the inspection of a heater treaterpump, is displayed as the digitized document 404 in a scrolled window,with a widget palette 406 displayed in a separate window in the top,right portion of FIG. 4. In this example, an interaction designer hadalready selected a general-purpose Section widget 408 from the palette406 and superimposed it over the first section of the digitized document404, the section titled “Purpose”, and had configured some of theproperties/attributes of the selected widget, displayed in a window 410which is located below the widget palette display in FIG. 4. Forexample, the selected Section widget 408 is configured to have a“Section Name” attribute with a value of “Purpose” and to be a“Collapsible” widget meaning that the widget may be displayed collapsedor expanded both within IAT 402, when defining interactions, and throughIPT 702 (see FIG. 7), when interactions are being performed. The“Layering Effect” of the selected Section widget 408 is set to“Superimpose” so the widget 408 is superimposed (being transparent) overthe “Purpose” section vs being overlaid (which would cover the contentblock under it). The selected Section widget's sequence number (underSequence->Current) is set to “1.0”, the previous widget's sequencenumber (Sequence->Previous) is not set since the selected widget is thefirst widget, and the sequence number of the next widget to be performed(Sequence->Next), at job performance time, is set to “2.0” (i.e.Sequence->Current+1) by default. The interaction designer may change theSequence->Next attribute to point to a widget with a different sequencenumber or to use a conditional widget to dynamically determine the“Next” widget to automatically navigate to at job performance time, aswill be described later. Furthermore, the selected widget is configurednot to require an acknowledgement by the user of IPT 702 at jobperformance time, before the user may mark the widget as completed andproceed to the next widget containing the next content block to beconsidered/performed. The selected Section widget, as it may be the casewith all interaction widgets, also snags the contents of the section orthe content block contained within it as part of its metadata togetherwith the value of its attributes as configured by the interactiondesigner. One with ordinary skill in the relevant art will acknowledgethat considering the embodiments described in this disclosure, there arevarious different interaction attributes and IAT 402 designs that may beconceived utilizing conventional methods of implementation available inthe industry.

As illustrated in FIG. 1, step 107, if the digitized document is of arich content format, such as PDF, the IAT 402 also captures the contentof the content block contained by the selected widget as illustrated atstep 107.1. In FIG. 1, step 109, the interaction designer configures therelevant user-computer interaction provided by the widget to bedynamically enforced when the contained content block of the digitizeddocument is being followed at job performance time.

As illustrated in FIG. 1, step 108, when a superimposed/overlaid widgetneeds to occupy a space that is larger than the space provided by thespecific area of the displayed digitized document that is beingcontained by the widget, the displayed digitized document is modified,see steps 108.1 and 108.2, to add the required space and the modifiedversion is displayed instead of the original. As an example, when aCaution widget is superimposed over a Caution section of a digitizeddocument, requiring acknowledgment at job performance time, if there isno room between the Caution section and the next section to insert anacknowledgment button, a modified version of the digitized document,containing the required added space will be created and the modifiedversion will replace the last displayed digitized document in theassociated display. One way to accomplish the modification is to splitthe digitized document into two documents where the first documentcontains all the contents up to the end of said Caution section and thesecond document contains all the content of the digitized document afterthe Caution section and then to add the desired blank space to the endof the first document and merge back the first and the second documentwhile readjusting all the coordinates of the widgets alreadysuperimposed/overlaid in locations of the second document to reflect thenew coordinates resulting from the addition of the blank space anddisplaying the merged document instead of the last displayed digitizeddocument.

Once the interaction designer has repeated the steps 104 through 110 ofFIG. 1, for each desired content block or sub-content block (i.e.content within a content block already contained within an interfacewidget), repeating to contain all the desired content/sub-content blocksof the displayed digitized document, the interaction designer moves tostep 111 to define the order of appearance/activation of the widgets atjob performance time. One of the attributes in common across all widgetsis the attribute that defines the order of appearance/activation acrosscontent types contained by the widgets, for example through a sequencenumber attribute, resulting in computer-guided navigation at jobperformance time. The order of appearance may be defined by staticallylinking/pointing one widget to another, for example by one widgetreferencing another widget's sequence number as its next widget tonavigate to, or conditionally and based on dynamically gathered data atjob performance type, one widget may point to one or more possible nextwidgets depending on dynamic evaluation of a conditional branch. Forexample, if a condition is evaluated to true, a widget that currentlycompleted its interaction may point to the sequence number of one nextwidget, and if the condition is evaluated to be false, to another nextwidget. These expressions may be expressed through special types ofinteraction widgets such as a Conditional Action widget described inthis disclosure.

FIG. 5 shows the same digitized document 404 as FIG. 4, with moresuperimposed and linked interaction widgets, where one of theinteraction widgets contains child interaction widgets, such as childinteraction widget 412.1, (or interaction sub-widgets) that signifysteps within the parent widget and with more space being added to thedisplayed digitized document to accommodate space for the Acknowledgmentrelated interactions needed for some of the superimposed interactionwidgets configured to require acknowledgment. Note that the metadata foran interaction sub-widget, may not be required to contain the snaggedimage or the content contained within said sub-widget, since the parentwidget already contains said content as part of its metadata and theposition of sub-widget may be stored relative to the position of itscontaining parent widget.

As shown in FIG. 5, the “Purpose” section is encapsulated by aninterface widget 408 as also shown by FIG. 4 and explained earlier.Explaining further, the Section interface widget 414 associated with thecontent block in the section titled “Scope” was configured to requireacknowledgment of consideration/performance at job performance time, andsince there was not enough space to insert an “Acknowledge” button (as ameans for enforcing the acknowledgment at job performance time), inbetween the sections titled “Scope” and “Responsibilities” in theoriginal displayed digitized document 404 within the Section interfacewidget 414 associated to the “Scope”, more space was added (refer toFIG. 1, step 108 and the explanation of the step provided earlier).Furthermore, the sequence numbers (signified by the Sequence->Currentand attribute shown in FIG. 4) associated with the interface widgets, inthis case, corresponds to the numbering of the sections/sub-sections ofthe digitized document. Accordingly, in this case, the Previous and Nextwidget references for each widget corresponds to the previous and nextsection's numbering. Furthermore, as demonstrated in FIG. 5, theinteraction Section widget 412 associated with the fourth section of thedisplayed digitized document, titled “PRECAUTIONS & LIMITATIONS”,contains interaction sub-widgets corresponding to each step of saidsection. These sub-widgets have hierarchical sequence numbers such as“4.1”, “4.2”, assigned based on the sequence number of their parentwidget.

Once the interaction designer has contained all the desired content (orsub-content) blocks through corresponding interaction widgets (orsub-widgets) and finished the configuration and static or dynamic andconditional sequencing of the widgets, information associated with allthe widgets is converted to structured metadata and saved as IIDM (orother appropriate data) as illustrated in FIG. 1 steps 112 and 113. Theinformation includes, but is not limited to, the superimposed/overlaidposition of the graphical widgets with respect to the displayeddigitized document, a snagged image corresponding to the content blockof the digitized document that is contained by each widget (note thatthis may not be needed for interaction sub-widgets as noted earlier),all the original content of the corresponding content block contained bythe widget if available and needed, and all of the widget properties,conditional and sequencing information and other attributes. Also, acopy of the last displayed version of the displayed digitized documentis saved with the metadata.

Yet, in another embodiment of the disclosure, generally, a set ofinteraction widgets are provided to correspond, more specifically, tothe sections within typical procedure/instructions, including but notlimited to, Purpose, Scope, Responsibilities, Precautions andLimitations, Prerequisites, Parts and Materials, Instructions,References, and Attachments or a General Content widget that may containany content block. Alternatively, or in conjunction with suchspecialized widgets associated with each typical section or contentblock, a general-purpose Section widget (as shown in FIG. 5), with a“Section Name” as one of its configurable attributes, may be providedthat may be generically mapped to any section within the digitizeddocument by the interaction designer and its “Section Name” attributecan be set to the title of the mapped section.

For example, if the Section widget is mapped to a section of thedigitized document titled “Purpose”, then the “Section Name” attributeof the widget may be set to “Purpose” by the interaction designer (or asdiscussed later through automation). Furthermore, another set of widgets(or sub-widgets) may be pre-configured to associate to the typicalsubsections under the Instructions section, including but not limitedto, Step or Action Step, Time Dependent Step, Continuous Action Step,Note, Caution, Warning, Conditional Action, Branch (or Go to), ComponentVerification, Reference (or Refer to), Bulleted list, Figure, Table,Critical Step, and Hold/Inspection Point or Independent Verification.These typed widgets are examples of widgets that may be available in thedisplayed “Interaction Widget Palette” referred to in FIG. 1, step102.2.

In some embodiments, a general Section widget with general and furtherconfigurable interaction properties is mapped to a section of a contentblock when no explicit keyword association is configured to match with atitle of what is assumed to be a section of the content block. One ofthe attributes of the widget can capture the title, if any, of thesection.

In some embodiments, a General Container or Content widget with generaland further configurable interaction properties is mapped to a contentblock when no explicit keyword association is configured and matchedwith the title of a content block. One of the attributes of such awidget is designed to capture the title, if any, of the content block.

Yet, in another embodiment of the disclosure, generally, an interactionwidget associated with some content type may contain child interactionwidgets and those child interaction widgets may contain other childinteraction widgets to any level of depth. One approach to capturing theorder of widget navigation is to associate hierarchical sequence numbersto widgets. For example, if the interaction widget associated with anInstruction section of the digitized document has a sequence number of5, the first child Step interaction widget corresponding to the firststep of the Instruction widget will have a sequence number of 5.1, thesecond child Step widget will have a sequence number of 5.2 and so on.This scheme for sequence number can accommodate any level of widgetparent/child relationship. The content type associated with a parentinteraction widget will be marked completed automatically when all itsrelevant child interaction widgets are marked completed.

Yet, in another embodiment of the disclosure, generally, the set ofavailable widgets, their interaction and interface behavior and theirassociation with the sections and content blocks ofprocedure/instructions may be configuration-driven. One way to approachthe configuration mechanism is to introduce a superset widget thatprovides a superset of capabilities, for all possible interactions, andproperties to turn those capabilities on/off, say with defaults set tooff, and then to allow the user to define any number of named customwidgets each with a relevant subset of those capabilities, by turning oneach desired capability through configuration. For example, the user maydefine a widget named “Caution” widget, using an instance of saidsuperset widget, and only turn on the “acknowledgment required” andplacekeeping interaction capabilities of the associated instance ofsuperset widget through an XML-based configuration mechanism.

Yet, in another embodiment of the disclosure, generally, a dynamic Tableof Contents (TOC) of all the widgets superimposed/overlaid on top of thedigitized document is represented on a display, in the order defined bythe links across the widgets, through their sequence numbers, or othermeans of sequencing, and with each entry in the TOC uniquelyrepresenting a superimposed/overlaid widget, in a manner that the TOCcan be used to navigate to a particular widget in the display thatcontains the content blocks of the digitized document through hyperlinksor a similar manner both within IAT 402 and IIP.

Yet, in another embodiment of the disclosure, generally, the interactiondesigner may define named views and then associate eachsuperimposed/overlaid widget to one or more of said named views throughan attribute provided on the widget for defining such associations suchthat, at job performance time, the user of the IPT 702 is able to switchto different views, where each selected view only displays the contentof the widgets associated to said view.

Yet, in another embodiment of the disclosure, generally, the interactiondesigner can divide the digitized document into two parts: 1) FrontMatter (usually containing sections such as Purpose, Scope,Responsibilities, Precautions and Limitations, and Prerequisites) vsBody (usually containing the procedure or instructions). And thenrequire the person performing the job, at job performance time, toacknowledge the Front Matter before he/she may proceed to the Body.Alternatively, the IAT 402 may provide a widget designed to containmultiple sections of the digitized document, say Section widget, in thiscase containing all the sections of the Front Matter, and turn on theacknowledgment interaction for that widget and link it to the firstwidget of the Body.

Yet, in another embodiment of the disclosure, generally, when thedigitized document is expressed in a rich content type, by extractingand storing the original contents of the content block contained by awidget with the associated widget metadata, the procedure/instructionsand the user-computer interaction widgets associated to its sections maybe also presented with a different form factor on any device such as asmart phone or smart glasses at job performance time, withoutrelying/using the digitized document in the background, and by firstdetermining the position of the next widget with the smallest sequencenumber on the device display and then displaying the content relative tothe position of the containing widget on the display, then based onwidget sequence numbers, selecting a next widget and determining itsposition relative to the first widget and then displaying its contentrelative to the position of its containing widget, until all contentblocks are displayed relative to the position of their containingwidgets on the display. Then, placing the first widget on the display ofa computational device to signify the current content block underperformance and provide placekeeping and other interaction through thedisplay of the widgets as work progresses at job performance time.

Yet, in another embodiment of the disclosure, generally, the widgetsassociated with some select sections of the procedure/instructions maybe superimposed/overlaid on top of the saved digitized documents and theperformance of some other select sections, including but not limited tosteps within select instructions, maybe transferred to another device,such as a smart glass, without relying/using the digitized document inthe background.

Yet, in one embodiment of this disclosure, generally, a widgetassociated with a Conditional Action may support branching based on anIF/WHEN [condition], in order to link the widget to another specificwidget thus causing automatic navigation to the specific widget at jobperformance time. The condition may be expressed as a binary or a seriesof binary questions, for example with Yes/No answers, linked throughlogical operators such as AND/OR/NOT or EITHER/OR, or any number oflogical operators and conditions. A Conditional Action Widget mayreference data that was previously entered by a user to make adetermination of whether a widget and its content should be displayed atall when a prior completed widget pointing to it is marked as completedor Not Applicable (N/A).

FIG. 6 shows an example of the use of a Conditional Action step widget416.1.3, within one embodiment of IAT 402, to define automatic branchingand navigation to the next interaction widget based on dynamic userinput later provided at job performance time within the IPT 702. Asshown, the interaction widget 416 associated with the “INSTRUCTIONS”section of the displayed digitized document, contains a Step interactionsub-widget 416.1 associated to the first step of the instructionssection. The Step interaction sub-widget 416.1, itself, contains fiveinteraction Step sub-widgets 416.1.1, 416.1.2, 416.1.3, 416.1.4, and416.1.5. The sub-widget 416.1.3 associated to the third sub-step (withsequence number 6.1.3) is a Conditional Action step interaction widgetoverlaid on top of the corresponding content block of the displayeddigitized document. The original instruction in the digitized documentread “If Pump cover is not installed, Go To Section 6.2”, however,because the Conditional Action widget was overlaid (vs superimposed),the original content block is covered by the widget, and the interactiondesigner configures the Conditional Action widget to read: “Is Pumpcover installed?”, and provides two buttons corresponding to twopossible binary answers of “Yes” or “No” and in this case configures the“Yes” answer to go to the widget 416.1.4 with the next sequence numberand “No” to branch to a widget 416.2 with sequence number 6.2. At jobperformance time, as shown in FIG. 7 and explained later in thisdisclosure, if the user presses the button label with “Yes”, the IPT 702displays/activates the widget 416.1.4 associated with the next sequencenumber, 6.1.4, and if the “No” is selected, the IPT 702 branches anddisplays/activates the widget 416.2 with 6.2 (and, in this case, alsoits first child widget 416.2.1 at sequence number 6.2.1.

Yet, in one embodiment of this disclosure, generally, a widgetassociated to a Repeat section is provided to support two basic forms ofrepeating: 1) by embedding a Conditional Action widget that causesautomatic navigation through branching to a prior step/widget, 2) bybeing configured to repeat a specified number of times. In any case, atjob performance time, the user will be able to see the originalperformance results and its associated data for each step withoutobscuring the use of the steps for the repeat, for example, throughmultiple tabs, where each tab is associated to an iteration of therepeat with the current iteration in the first tab. When a Repeatsection is complete, IPT 702 either navigates the user back to theorigin or to the next sequential widget, depending on the rulesconfigured through IAT 402 or the structured metadata.

Yet, in another embodiment of the disclosure, generally, an interactionwidget, for example one associated with a simple Action Step, may beconfigured to enforce time requirements. These requirements may include,but are not limited to, minimum time, maximum time, or a time range forthe performance of the widget. At job performance time, the user isprovided with a clock/stopwatch that he/she may manuallystart/stop/reset, where the clock changes visual state when the timerequirement is met, which then the user may mark the step as completed.

Yet, in one embodiment of this disclosure, generally, an interactionwidget, for example one associated with a simple Action Step, may beconfigured to require and enforce Concurrent Verification (CV), wheretwo users performing a step/section must be physically present together.In this case, both workers will perform the step together and sign off,through initials or digital signature sub-widgets, after the performanceof the step/section is completed and before they are allowed to proceedto the next step/section of the procedure/instructions. IPT 702 mayvisually render a CV step/section distinctly to ensure that the mainuser and the concurrent verifier are both present before performing thestep. Similarly, an interaction widget may be configured to require andenforce Independent Verification (IV), where the only difference is thatthe verifier does not have to be present at the time the step/section isbeing performed. When an interaction widget configured as IV is markedas complete, the IPT 702 will not yet progress to the next step, insteadit sends a notification, maybe through an embedded Notificationsub-widget, to the independent verifier with a link to the IV widgetrequiring verification, and only then will IPT 702 navigate the user tothe widget associated with the next step/section.

Yet, another embodiment of the disclosure relates, generally, to IAT 402providing procedure/instructions authoring capabilities through apalette 406 of interaction widgets used for authoring newprocedures/instructions, containing widgets types corresponding to thewidgets types used for superimposing/overlaying on top of the existingwritten procedure/instructions, except that these procedure/instructionsauthoring widgets provide means for authoring and creating new contentblocks. Widget instances from this palette may be connected andcombined, through sequence numbering, with the widgets used forsuperimposing interaction on top of existing content blocks.

Yet, in one embodiment of this disclosure, generally, any interactionwidget may be added to a set of reusable widget templates, wherereusable widgets templates are grouped by their type, for examplereusable widget templates of type Caution, and made available within theIAT 402 so they may be combined with other interaction widgets. When awidget is made reusable, the sequence number of the corresponding widgettemplate becomes dynamic and, in some such embodiments, it is onlyassigned when an instance of the widget template is connected orembedded in other interaction widgets based on the sequence number ofthe connected or containing widget and if the widget template containschild widgets, the hierarchical sequence number of the child widgets areadapted to the parent widget.

Yet, another embodiment of the disclosure relates, generally, to thebidirectional integration of data and automatic dispatch of applicationfunctions from other applications, such as enterprise asset or workmanagement systems, to/from an existing written procedure/instructions,instructional manual or fillable form, in PDF or any digitized binaryformat (the digitized document), driven by the selection of a softwareservice interface with typed input/outputs, including but not limitedto, standards-based Web Services, whereby selecting an input or outputof a service operation, based on its type, an interface widget,configured to be associated to that type, is provided within the IAT 402that may be overlaid or superimposed on any part of the content block ofthe digitized document, where said any part of content block may beencapsulated within an existing interaction widget. IAT 402 saves theinterface widget together with information regarding the selectedsoftware service and its input/outputs as structured metadata in IIDM.At job performance time, when an interface widget associated with theinput of a service is activated, the user will be prompted to providethe input within the area covered by said input interface widget andonce all inputs to a service are provided, the service will be eitherinvoked automatically, or its execution may be tied to a user triggerevent such as the press of a button through a button widget. When awidget associated with a service output is activated, at job performancetime, the area of the content block covered by the interface widgetassociated with that output will be automatically populated with theassociated output data from the execution of the service. In the casewhich both input(s) and output(s) of a service are associated withcontent blocks through multiple interface widgets, the service may beexecuted, only after all input data is entered by the user. If aninterface widget is contained within an interaction widget, it willactivate only when the containing interaction widget displays.

There are a number of differences between this method of integration andother prior art methods. One major difference between this method andother previous methods is that the previous methods either allow theuser to place an input data widget anywhere on the display or to relatethe input data widget to a fixed, inflexible location of an underlyingdigitized document. Instead, in this method, an input widget, used totake user input at performance time, or an output widget, used topopulate a portion of the display with data from other systems, isplaced relative to the position of a containing interaction widget, orany container widget used to encapsulate and contain a content block ofthe digitized document, and regardless of the digitized document andwithout requiring the digitized document at performance time. Anothernotable difference is that the inputs and outputs of a software service,such as a standards-based Web Service (e.g., WSDL/SOAP or REST), areimplicitly associated to input/output widgets at design time.Furthermore, at performance time the associated software services areautomatically executed resulting in direct integration with othersoftware applications through their business logic layer, as opposed tosimply saving the data in a database only to be integrated later. Themethod of containing the input/output interface widgets within otherlinked interaction/container widgets supports new capabilities that notonly does not require the inflexible association of the input/outputwidget to a hardcoded location of the digitized document, but alsoprovides means for interactive computer-guided navigation at jobperformance time.

Yet, another embodiment of the disclosure relates, generally, to a DataInterface Widget for simple data entry by the user, without being tiedto the inputs/outputs of a software service. The Data Interface widgetmay be superimposed/overlaid on any part of a content block that isalready contained within any other widget and the data entered throughsuch widget is made available throughout the interactiveinstruction/procedure/form to be referenced by all widgets with sequencenumbers greater than the sequence number of the Data Interface Widget orits containing widget. Furthermore, the users of IAT 402 and IPT 702 mayenter constants in interface widgets to support computation (ex min/maxvalues). The location of a Data Interface Widget is saved relative toits containing widget just as with other interaction or interfacesub-widgets.

FIG. 2 illustrates a flowchart for one embodiment of the method fordefining application and data integration through superimpose/overlay ofinterface widgets on top of content blocks, or portions thereof, ofwritten procedure/instructions, instructional manuals and/or fillableforms, resulting in structured metadata definitions saved as IIDM. FIG.2 illustrates obtaining a digitized document at step 201 and displayingthe digitized document at step 202. While FIG. 2, step 203, refers to ageneral “container widget” that encapsulates and contains (through asnagged image or capturing the contents of) a content block (step 205)of the displayed digitized document such that it can be used independentfrom the digitized document at job performance time, an “interfacewidget” described in earlier embodiments of this disclosure qualifies asa container widget and thus the process described in FIG. 1 and FIG. 2may be combined to address both requirements for user-computerinteraction and application integration in a single process. FIG. 2,step 206 accommodates creating interface widgets both based on softwareservices (see steps 206.1, 206.2, and 206.3) as described earlier orcreating or selecting interface widgets (see step 207), referred to assimple Data Interface widgets, that do not require mapping toinputs/outputs of a software service but are meant to capture dynamicuser input data, during job performance, and make such data available toother widgets for reference, for example, in the evaluation of dynamicconditionals and for dynamically branching to the next widget.Regardless of the type of interface widget, in FIG. 2 steps 208 and 209,the interaction designer superimposes/overlays the interface widget overan area of the content block contained within a container/interactionwidget and the relative position of the interface widget with respect tothe container/interaction widget is added to the widget metadata.

As illustrated in FIG. 2, step 210, if the interface widget needs tooccupy a space that is larger than the space provided by the specificarea of the displayed digitized document that is being contained by itscontaining container/interface widget, the digitized document may bemodified (see step 210.1 and 210.2) to add more space following aprocess as explained earlier for FIG. 1, step 108. Attributes of aninterface widget are configured at step 211. FIG. 2, step 212illustrates how a container/interaction widget may contain one or morechild/sub widgets. One can modify the process illustrated in FIG. 2 toallow interface widgets to contain other interface widgets or to allowcontainer/interaction widgets to contain other container/interactionwidgets. As illustrated at step 213, portions of the process can berepeated for other content blocks. The process illustrated in FIG. 2 isconcluded in steps 214, 215 and 216 where widgets are sequenced, and allwidget information and attributes are translated into IIDM and saved.

Yet, in one embodiment of this disclosure, generally, a configurableNotification interface widget is provided to cause the automaticpublication of a notification to the person performing the job and/or toother users and applications through the dispatch of a software serviceassociated to said widget. The Notification widget may be configured toevaluate dynamic input/output data. When another person is notified, thenotification message provides a link such that the person may launch aninstance of the interactive procedure/instructions/form in an instanceof the IPT 702 and automatically navigate to the same place where thenotification was originated.

Yet, in one embodiment of this disclosure, generally, a configurableEquipment interface widget is provided to integrate events and data fromspecific Internet of Things (IoT) enabled equipment or to cause suchequipment to perform a specific action at job performance time. TheEquipment widget can be configured to connect to a specific equipmentthrough its unique identifier, which may be a URL, in order to invokespecific API's on the processor of that equipment.

One of ordinary skill in the related art, may easily combine theprocesses of FIG. 1 and FIG. 2 to create a single IIDM metadatacontaining definitions of both interface widgets and interaction widgetsto provide interactions and well as integrations at job performance timefor a single digitized document that represents writtenprocedure/instructions, instructional manuals and/or fillable forms.This may be accomplished, through one embodiment of this disclosure,generally, using interaction widgets defined through the processillustrated in FIG. 1 as “container widgets” referred to in Step 203 ofFIG. 2. This can be simply accomplished by replacing Steps 201, 202,203, 204 and 205 in FIG. 2, with Steps 101 through 110 of FIG. 1. Suchthat, once all interface widgets are defined, interaction widgets may beadded inside of the interface widgets. There are other ways of combiningthe processes illustrated in FIG. 1 & FIG. 2. For example, by insertingsteps 206 through 212 of FIG. 2, in between steps 109 and 110 of FIG. 1such that the creation of interface widgets and interaction widgets maybe intermingled. Furthermore, one can create variations of the steps inFIG. 1 and/or FIG. 2, or any flowchart that is derived by combining theprocesses in FIG. 1 and FIG. 2 into a single process resulting in asingle IIDM that can be applied to the digitized document. For example,one variation may allow some interface widgets to be placed inside ofother interface widgets or their sub-widgets.

In one or more embodiment of this disclosure, at job performance time,the IPT 702 interprets the saved IIDM metadata resulting from theprocess in FIG. 1 and/or FIG. 2 or any new process resulting fromcombining the two processes, as mentioned earlier, and applying thecombination to the digitized document. In one embodiment, the IPT 702may display the content blocks encapsulated by interaction/containerwidgets through any of the following means, or other appropriatemeans: 1) by displaying a copy of the saved version of the digitizeddocument in a scrolled window and providing placekeeping andinteractions/integrations by superimposing/overlaying the firstinteraction/container widget corresponding to the first containedcontent block of the digitized document, based on the associatedcoordinates, and sequencing and other information saved in the metadata,and then when current widget is marked as completed, displaying the nextinteraction/container widget in the sequence and so on and so forthuntil the last widget is displayed and its interactions are performed.2) By displaying all the snagged images of the content blocks associatedwith each widget in the sequence the widgets are linked and relative tothe simulated placement of the first widget (without the display of thewidget itself) on an empty computer display/window and then thesimulated placement of other widgets relative to the first widget andbased on their sequence number, without displaying or relying on thesaved digitized document, and then superimposing the first widget overthe already displayed content block contained by the first widget toactivate the widget and signify the initial content block underperformance, and when said initial content block is marked as complete,to display/activate the next relevant widget to signify the next contentblock to be performed, and repeat the process until all relevant widgetsare displayed and marked as completed. 3) Similar to 2 above but,instead of displaying the image, to display the original contentcontained by the widget optionally in different form factors appropriatefor the display on different computing devices. Regardless of the meansof display, at job performance time, the widgets displayed based on thestructured metadata are used to provide computer-guidance andapplication integration and to dynamically advance the user to the nextsection of the procedure/instructions relevant to the job.

Yet, in another embodiment of this disclosure, FIG. 3 relates to aflowchart of a method to perform, user-computer interaction andcomputer-aided navigation as well as application and data integration atjob performance time, based on the structured metadata IIDM resultingfrom combining the processes described in FIG. 1 and FIG. 2 and applyingthe combined process to the digitized document. In step 301 of FIG. 3,said resulting IIDM is imported and loaded into computer memory intodata structures (that may include hash maps, linked lists, arrays or acombination of other conventional data structures) that are optimizedfor the implementation of the process in FIG. 3. FIG. 3, step 302,relates to displaying all the content blocks contained by all theinterface/container widgets, in the order of their sequence number,starting from the smallest sequence number using either of the meansnumbered “2)” or “3)” for displaying content blocks encapsulated byinteraction/container widgets, described in the above paragraph withoutrequiring a copy of the digitized document. After all the content blocksare displayed in step 302, moving to step 303 of FIG. 3, the widgetassociated to the first displayed content block is activated/displayedaround said content block and marked as the “Current widget” (see step304) to provide a placekeeping function, process all it sub-widgets (seesteps 305, 306, 306.1, 306.2, 306.3, 306.4, 306.5, 306.6, 306.7, 306.8,306.9) if any, and also to enforce its defined interactions. Once thefirst content block is considered/performed by the user and its enforcedinteractions completed, the widget may be marked as completedautomatically (see step 307), for example, upon interaction requiringacknowledgment of the content block by the user, or manually by the userwhen automatically marking as completed is not an option. After thecompletion of the first widget, which includes the processing of itsrelevant sub-widgets described in step 306 and the sub-steps thereof ofFIG. 3, if any, IIP determines the next interface widget to beactivated/displayed, if any, and marks the widget as “Current widget”(see steps 308, 309, 310, and 311).

If the first or any subsequent widget that was marked as the “Currentwidget”, through performing step 304 of FIG. 3, does not have anychild/sub-widgets relevant to the job scenario being performed asdetermined in step 305 of FIG. 3, where such determination may be madebased on conditionally linked widgets that may result in skipping theperformance of all or some of its sub-widgets, if any, by branching toanother non-sequential widget, said “Current widget” completes all itsinteractions and is marked as completed through step 307, when saidwidget points to a next interaction widget (again such next widget maybe determined conditionally), as determined in step 308 of FIG. 3, thenext widget is displayed and activated, according to step 311 of FIG. 3,and the processing of that widget will loop back to step 304 of FIG. 3thus said next interaction widget will be marked as the “Currentwidget”.

If the first or any subsequent widget that was marked as the “Currentwidget”, through performing step 304 of FIG. 3, does have one or morerelevant child/sub-widgets, where such relevance may be determinedthrough dynamic conditional branching based on job data gathered so farthrough the process, as determined in step 305 of FIG. 3, all thesub-widgets of the “Current widget” and, in the case where anysub-widget is yet another interaction widget, all of its sub-widgets,and so on and so forth will be processed by the embodiment of IPT 702according to the step 306 of FIG. 3 together with its sub-steps. One canmodify the process described in FIG. 3, to allow processing of interfacewidgets that contain other interface widgets, to any level of depth.

In step 306.1 according to one embodiment of the disclosure illustratedin FIG. 3, each sub-widget of the “Current widget” is either aninterface widget or an interaction widget. Each relevant sub-widget ofthe “Current widget”, is processed through the sub-steps of step 306.For each relevant next sub-widget being processed, when the sub-widgetis not an interface widget, as determined in step 306.1, it is impliedthat the sub-widget is an interaction widget, in which case, theprocessing continues through step 309, to step 310 of FIG. 3, where the“Current widget” that is the parent interaction widget of the sub-widgetis placed on a Stack data structure so its processing continues afterits sub-widget is processed. Then, through step 311, the interactionsub-widget is activated/displayed and processing loops back to step 304,where the sub-widget becomes the “Current widget”. Using this process,interaction sub-widgets, within other interaction sub-widgets, and so onand so forth to any level of depth may be processed. The parent widgetof an interaction sub-widget is pushed on a processing “Stack”, suchthat it can be ‘popped’ after its interaction sub-widget completes itsprocessing through step 307.c of FIG. 3, to continue its processing asthe “Current widget”. The use of a Stack data structure accommodatesembedded interaction sub-widgets to any level of depth. To keep theprocess flowchart in FIG. 3 easy to understand, many conventionaldetails associated with error-handling or special cases have beenomitted. One with ordinary skill in the art will acknowledge that suchdetail can be addressed through conventional data structures andprocesses common in the relevant art.

Referring once again to FIG. 3, when a relevant sub-widget is aninterface widget, as determined in step 306.1, if the definition of thatsub-widget is not tied to a software service, as determined in step306.2, the sub-widget is a Data Interface widget that based on itsdefined type is used to either gathered data by prompting the user orused for displaying and will be processed through step 306.6. If thesub-widget is tied to a software service, as determined in step 306.2,the processing of the interface sub-widget moves to step 306.3 where thewidget is determined to be either tied to the input(s) or output(s) of asoftware service. If it is tied to the output(s), through step 306.4,the associated software service is executed, if it was not alreadyexecuted, and the associated service output data is displayed throughthe sub-widget within the area of the content block occupied by thesub-widget. Otherwise, if the sub-widget is tied to input(s) of theassociated service, in step 306.5, the user is prompted for thereferenced input data through the associated sub-widget and the serviceis executed when the last input data is gathered through an interfacesub-widget. All the steps 306.4, 306.5, and 306.6 will go through step306.7 of FIG. 3, where all input and/or output data gathered will bemade available to be referenced and accessed by all proceeding widgetsand sub-widgets and processing moves to step 306.8, where the processingcontinues to the next sub-widget, if any, through step 306, and whereall sub-widgets of the “Current widget” are processed, processing moveto step 307 where all remaining interactions of the “Current widget” arecompleted and the widget is marked as complete and if there are anyparent widgets on the Stack, each parent widget is ‘popped’ in order andits processing is resumed as the “Current Widget”. The processillustrated in FIG. 3, does not address processing of interface widgetsas sub-widgets of other interface widgets. One with ordinary skill inthe art acknowledges that this process may be easily modified toaccommodate such processing.

If the first or any subsequent widget that was marked as the “Currentwidget” performing its interactions does not have any child/sub-widgetsas determined in step 305 of FIG. 3, and it does not point to a nextinteraction widget, as determined in step 308 of FIG. 3, there are nomore widgets to be processed and thus all relevant interactions andintegrations associated with the job are already performed, whereprocessing moves to step 312 which signifies the end of processing ofIIDM by this embodiment of IPT 702 according to FIG. 3.

FIG. 7 shows an example of an embodiment of the Interaction PerformanceTool (IPT) 702, performing user-computer interaction, computer-guidednavigation and placekeeping as shown (and partially defined) in FIG. 6.As shown in FIG. 7, the performance of steps 6.1 and 6.2 were alreadyconfirmed by the user through their associated Action Step interactionwidgets and thus they are marked as “Completed” by the IPT. Furthermore,the user has selected the button labeled “No” when the overlaid widget416.1.3 associated with step 6.1.3 was activated/displayed, andtherefore, the widgets 416.1.4 and 416.1.5 associated with steps 6.1.4and 6.1.5 were automatically marked as “N/A” and the configured textsupplied for the “N/A” state of the corresponding widgets and the widget416.2.1 with sequence number 6.2.1, corresponding to section 6.2.1 ofthe document is automatically activated/displayed and navigated to, asthe next relevant step that needs performance and thus performed by theuser.

Yet, in another embodiment of the disclosure, generally, any of thewidgets, regardless of type, provide a set of general interactioncapabilities at job performance time through IPT 702 some of which maybe configured through the TAT 402 or the structured metadata associatedwith interaction widgets and their sub-widgets. These capabilities mayinclude, but are not limited to: a) the ability to add comments to eachwidget, through say a Comment sub-widget that can be placed andrelocated to any area within said interaction widget by the user, b) theability for a user to provide feedback/change-request on a content-blockassociated with an interaction widget, for example through a Feedbacksub-widget similar to the Comment sub-widget, such that theprocedure/instructions writer may consider the feedback at a later time,c) the ability to flag any interaction widget to be added/removedto/from bookmarks for easy navigation, d) the ability to expand orcollapse all or some of the interaction widgets, including their childinteraction widgets, e) the ability to capture media (ex. video,pictures, sound) through Media sub-widgets that can be associated withany interaction widget, f) the ability for the user to overwrite thesequence in which the computer navigation is designed to follow based onthe sequence numbers associated with the interaction widgets orconditional branching, under certain circumstances, however requiringjustification in the form of enforced Comment/Feedback and initials ofthe user, g) provide the ability to jump to the current step or sectionof the interactive digitized document with one click, h) provide theability for the user to see progress status of the completed stepsagainst the total required steps local to interaction widgets associatedwith an instructions widget and its child widgets as well as overallprogress with respect to the entire procedure/instructions/form. Notethat using IAT 402, the person defining the interactions through widgetscan specify what percentage or how many units of work is associated witheach widget relative to the overall work and for sub-widgets relative tothe parent widget, i) the ability to enforce the navigation of thewidgets associated with the numbered sections and steps of instruction,and the sub-widgets associated with bulleted sections in an interactiveprocedure/instructions in order, such that sequential performance isenforced and a later step/widget is not allowed to be marked complete(or N/A if not applicable) until the prerequisite step/widget is markedcomplete, or out of sequence, such that the associated steps/widgets canbe performed in any order and marked as complete (or N/A if notapplicable, based on the selected widget properties designated throughIAT 402 or expressed in the structured metadata associated with aprocedure/instructions/form, j) the ability to automatically mark aparent widget complete when all its child widgets are dispositioned ormarked as complete, k) the ability to automatically provide centralizedmenus or Table of Contents for all widgets and sub-widgets associatedwith the sections and sub-sections as well as for the references andfigures such that the user may directly navigate to any of them throughthe menus, 1) the ability to support full-text search as well assearch/view by section type such as all Cautions or all the interfacewidgets used for data entry, through which search/view the user maydirectly navigate to the content of the widgets associated to the searchresults, m) the ability for the user to pause the performance of asection/step such that the associated widget timer freezes and no stepcan be dispositioned and the display becomes read only until resumed, n)ability to zoom in/out, pan and annotate figures/diagrams, o) to supportfour different states for each widget and sub-widget: Not-started,Under-performance, Completed or N/A (Not-applicable), p) to highlightCritical steps distinctly, q) to provide an indexed view for eachwidget/sub-widget type, such as Caution, Comment, Feedback, Media orother widgets and sub-widgets associated to each content-block, so itcan be reviewed separately, and provide easy navigation to theinteraction widget against which the comment was entered and thus to thecontent-block contained by that interaction widget.

Yet, another embodiment of the disclosure relates, generally, tocapturing and aggregating information about the dynamic, user-computerinteraction during job performance to provide job performancemonitoring, statistics and analytics. Examples of such data includes,but is not limited to, location of job performance through device GPScoordinates, the elapsed time between the completion of each step ofinstruction, and other dynamic data resulting from user-computerinteraction throughout job performance.

Yet, another embodiment of the disclosure relates, generally, toautomatic generation of IIDM by programmatically reading a stream ofdigitized written procedure/instructions through a computer processorand into a computer memory, for example in PDF format, and parsing andanalyzing the contents and dispatching subroutines to programmaticallysuperimpose widget metadata corresponding to typicalprocedure/instructions labels encountered where the label to widgetassociations may be configuration-driven. For example, when a Cautionsection is encountered in the stream, a rectangular Caution interfacewidget is automatically created with its dimensions corresponding to thearea needed to sufficiently contain the content type of the associatedCaution section, and a sequence number that is automatically assignedconsidering the sequence number of the last automatically generatedinterface widget then saved within the IIDM.

Yet, another embodiment of the disclosure relates, generally, to addingpalette(s) of widgets to IAT 402, designed for encapsulating commandsthat may drive one or more computerized machine(s) capable of carryingout a complex series of actions, where each command drives one or moreaction(s), and relates to a step of an instruction encapsulated with awidget from said additional palette(s). With the practice of thisembodiment, instead of a user performing one or more of the steps withinan instruction, one or more computerized machine(s) may be automaticallydriven to perform said steps(s).

Yet, another embodiment of the disclosure relates, generally, toautomatic adjustment of the structured metadata in the IIDM associatedwith an original digitized document, based on a revised version of theoriginal digitized document, by programmatically comparing the revisedversion with the original and identifying the sections deleted and addedto/from the original document and then adjusting the IIDM to delete thewidgets associated with the deleted sections and to re-compute thecoordinates of the remaining widgets in IIDM to account for the newcoordinates of the corresponding widgets.

Further, the methods illustrated above may be practiced by a computersystem including one or more processors and computer-readable media suchas computer memory. In particular, the computer memory may storecomputer-executable instructions that when executed by one or moreprocessors cause various functions to be performed, such as the actsrecited in the embodiments.

Embodiments of the present invention may comprise or utilize a specialpurpose or general-purpose computer including computer hardware, asdiscussed in greater detail below. Embodiments within the scope of thepresent invention also include physical and other computer-readablemedia for carrying or storing computer-executable instructions and/ordata structures. Such computer-readable media can be any available mediathat can be accessed by a general purpose or special purpose computersystem. Computer-readable media that store computer-executableinstructions are physical storage media. Computer-readable media thatcarry computer-executable instructions are transmission media. Thus, byway of example, and not limitation, embodiments of the invention cancomprise at least two distinctly different kinds of computer-readablemedia: physical computer-readable storage media and transmissioncomputer-readable media.

Physical computer-readable storage media includes RAM, ROM, EEPROM,CD-ROM or other optical disk storage (such as CDs, DVDs, etc.), magneticdisk storage or other magnetic storage devices, or any other mediumwhich can be used to store desired program code means in the form ofcomputer-executable instructions or data structures and which can beaccessed by a general purpose or special purpose computer.

A “network” is defined as one or more data links that enable thetransport of electronic data between computer systems and/or modulesand/or other electronic devices. When information is transferred orprovided over a network or another communications connection (eitherhardwired, wireless, or a combination of hardwired or wireless) to acomputer, the computer properly views the connection as a transmissionmedium. Transmissions media can include a network and/or data linkswhich can be used to carry desired program code means in the form ofcomputer-executable instructions or data structures and which can beaccessed by a general purpose or special purpose computer. Combinationsof the above are also included within the scope of computer-readablemedia.

Further, upon reaching various computer system components, program codemeans in the form of computer-executable instructions or data structurescan be transferred automatically from transmission computer-readablemedia to physical computer-readable storage media (or vice versa). Forexample, computer-executable instructions or data structures receivedover a network or data link can be buffered in RAM within a networkinterface module (e.g., a “NIC”), and then eventually transferred tocomputer system RANI and/or to less volatile computer-readable physicalstorage media at a computer system. Thus, computer-readable physicalstorage media can be included in computer system components that also(or even primarily) utilize transmission media.

Computer-executable instructions comprise, for example, instructions anddata which cause a general purpose computer, special purpose computer,or special purpose processing device to perform a certain function orgroup of functions. The computer-executable instructions may be, forexample, binaries, intermediate format instructions such as assemblylanguage, or even source code. Although the subject matter has beendescribed in language specific to structural features and/ormethodological acts, it is to be understood that the subject matterdefined in the appended claims is not necessarily limited to thedescribed features or acts described above. Rather, the describedfeatures and acts are disclosed as example forms of implementing theclaims.

Those skilled in the art will appreciate that the invention may bepracticed in network computing environments with many types of computersystem configurations, including, personal computers, desktop computers,laptop computers, message processors, hand-held devices, multi-processorsystems, microprocessor-based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, mobile telephones,PDAs, pagers, routers, switches, and the like. The invention may also bepracticed in distributed system environments where local and remotecomputer systems, which are linked (either by hardwired data links,wireless data links, or by a combination of hardwired and wireless datalinks) through a network, both perform tasks. In a distributed systemenvironment, program modules may be located in both local and remotememory storage devices.

Alternatively, or in addition, the functionality described herein can beperformed, at least in part, by one or more hardware logic components.For example, and without limitation, illustrative types of hardwarelogic components that can be used include Field-programmable Gate Arrays(FPGAs), Program-specific Integrated Circuits (ASICs), Program-specificStandard Products (ASSPs), System-on-a-chip systems (SOCs), ComplexProgrammable Logic Devices (CPLDs), etc.

The present invention may be embodied in other specific forms withoutdeparting from its spirit or characteristics. The described embodimentsare to be considered in all respects only as illustrative and notrestrictive. The scope of the invention is, therefore, indicated by theappended claims rather than by the foregoing description. All changeswhich come within the meaning and range of equivalency of the claims areto be embraced within their scope.

What is claimed is:
 1. In a computing system designed for having one ormore computers configured to implement a method to define user-computerinteraction and computer guided navigation for content blocks of awritten procedure or instructions, instructional manual or fillableform, with definitions used at job performance time to provide saidinteraction and navigation, a method comprising: obtaining a digitalversion of a digitized document, the digitized document comprising awritten procedure or instructions, an instructional manual or a fillableform; providing a set of typed dynamic graphical user interfaceinteraction widgets on an interactive computer user interface, whereeach widget is used to encapsulate and contain a content block, being alogical unit of content or a collection of such, from the digitizeddocument, and where each widget is designed to provide specificuser-computer interactions and computer guided navigation at a jobperformance time; superimposing or overlaying, in the user interface, afirst widget from said set of interaction widgets on top of or over acontent block of said digitized document thereby encapsulating andcontaining said content block; defining an order of appearance andactivation of each interaction widget relative to other widgets at saidjob performance time; configuring attributes of the widgets; and savingresulting widget information, comprising information about the contentblock contained within each widget together with its attributes, on acomputer-readable storage medium for execution or interpretation by oneor more processors when work associated with an encapsulated contentblock of the digitized document is being performed by an entity.
 2. Themethod of claim 1, further comprising: wherein providing a set of typeddynamic graphical user interface interaction widgets comprises providinga set of resizable interaction widgets corresponding to one or more ofsections, subsections, steps, or any content block representing one ormore logical units of the digitized document; receiving user inputselecting the widget corresponding to a section, subsection, step, orcontent block of the digitized document, from the set of widgets;receiving user input placing the selected widget on the user interfaceand resizing and moving it to surround and to contain said section,subsection, step, or content block; adding to a widget data structurefor the widget, at least one of the following: i) coordinates of thewidget relative to the displayed digitized document such that widgetsmay be superimposed or overlaid over a copy of the displayed digitizeddocument at job performance time, ii) a snagged image of a content blockcontained by the widget, such that at job performance time, contentblocks contained by widgets can be displayed irrespective of theposition of the widgets relative to a copy of the displayed digitizeddocument and without using the digitized document in the background, oriii) contents being contained by the widget corresponding to thecontained content block of the displayed digitized document, when thedigitized document is of rich-content type, such that the widget datastructure may be used later, to represent the corresponding contentblock of the digitized document in a different display form factor, fordifferent computer devices without relying-on or displaying theunderlying digitized document and by displaying the contents associatedwith the containing widgets based on the position of the containingwidgets relative to each other and then overlaying or superimposing thewidgets; setting attributes and properties of the widget as it relatesto look and feel and interaction choices relevant to the containedcontent block; defining the order of appearance of widgets, in order tosupport computer-guided navigation of the widgets at job performancetime, by providing means to sequence and link one widget to another,either statically or based on dynamically evaluated expressed conditionsand adding such sequencing information to the widget data structures;saving, on a persistent computer data store, a copy of the displayeddigitized document together with at least one of the following: i) theposition of the overlaid or superimposed widgets relative to thedisplayed digitized document, ii) the snagged image of the content ofthe content block contained by each widget, iii) content of the contentblock contained by each widget, iv) look & feel properties of eachwidget, v) interaction properties of each widget, or vi) sequencinginformation of each widget.
 3. The method of claim 1, wherein the set ofwidgets or their user-computer interaction properties comprise titles orheadings corresponding to at least one of: Purpose, Scope,Responsibilities, Precautions and Limitations, Prerequisites, Parts andMaterials, Instructions, References, or Attachments with one or morekeywords being associated to each title and the corresponding widgettype; wherein when an Instruction section is included, a subset ofwidgets types are configured to associate to subsection titles under theInstructions section, including at least one of, Step, Action Step, TimeDependent Step, Continuous Action Step, Note, Caution, Warning,Conditional Action, Branch, Component Verification, Reference, Bulletedlist, Figure, Table, Critical Step, Hold, or Inspection Point, with oneor more keywords being associated to each title and corresponding widgettype.
 4. The method of claim 1, further comprising displaying a dynamicTable of Contents (TOC) on a portion of a device display representingall the widgets superimposed or overlaid on top of the digitizeddocument in the order defined by the widget relationships or links orsequence numbers and with each entry in the TOC uniquely representing asuperimposed or overlaid widget in a manner that the TOC can be used tonavigate to a particular widget on a display containing the digitizeddocument.
 5. The method of claim 1, wherein each superimposed oroverlaid widget is associated with one or more named and indexed views,through a widget attribute, such that at job performance time, when theuser selects one of those views, only the content of the widgetsassociated to said view are displayed with a TOC containing only thewidgets associated to said view with navigation to the content block ofthe associated widgets.
 6. The method of claim 1, wherein generalconfigurable properties or sub-widgets contained within another widgetare provided across all widgets to support specific interaction behaviorat job performance time including at least one of: to support typed dataentry interface widgets corresponding to basic data types, complex dataentry widgets corresponding to tables, widgets for images, drawings,sound, video and other form media, such that all said widgets may beplaced inside of interaction widgets, or other data entry interfacewidgets; to view feedback associated to any select content block of thedigitized document contained by a widget, or feedback which wereprovided at job performance time by persons performing a job; to specifywhat percentage or how many units of work is associated with each widgetrelative to the overall work and for each sub-widget relative to theparent widget or the overall work such that at job performance time, theuser is enabled to see what percentage of work associated to widgets andsub-widgets is performed in relation to the total work associated to aparent widget or overall work; to require acknowledgment at jobperformance time, through a sub-widget or property, that the contentblock contained within a widget is read and conformed with, before theperson performing the procedure or instructions can continue to the nextcontent block contained by the next widget; or to cause a widget toappear collapsed or expanded, by default, at job performance time. 7.The method of claim 1, wherein a widget, including its content block andattributes except for the sequence designator attribute or an attributeused for sequencing, is added to a set of reusable widget templates,grouped by widget type, and a widget instantiated from a reusable widgettemplate is linked with other widgets taking on its sequence attributein the context of other linked widgets; and, where the reusable widgetinstance has child widgets, the child widgets' hierarchical sequencedesignators adapted to its parent widget's sequence attribute.
 8. Themethod of claim 1, further comprising, providing a set of procedure orinstructions authoring interaction widgets having widget types thatcorrespond to the widget types used for superimposing or overlayinginteractions on top of existing content blocks, and with the sameinteraction capabilities as their corresponding types, wherein theprocedure or instructions authoring interaction widgets are configuredto be used for authoring or defining new content blocks and areconfigured to be linked to an existing widget.
 9. The method of claim 3,wherein an interaction widget associated to a Step content block isplaced inside an interaction widget associated to an Instructionsection, as its child sub-widget, and linked with other widgetsincluding other Step widgets, with each Step widget having a series ofproperties and sub-widgets to support one or more of the following:declaring the order of execution of Steps, at job performance time, assequential, based on widget sequence numbers, or in an arbitrary order;indicating that completion of a step requires confirmation by the userand to provide a property or a sub-widget that may be placed inside aStep widget, for enforcing confirmation; a time-dependent step byautomatically providing a timer starting from when a step was entered,at job performance time, and automatically enforcing a time dependencyof a content block while providing the ability to pause and resume whenapplicable; indicating if the completion of a Step, at job performancetime, requires a workers initials and providing a sub-widget for theinitials that may be used, by placing it inside a Step widget;indicating if a Step is a Critical Step so that it may be rendered in avisually distinct way at job performance time; or indicating if a Steprequires two users to be physically present together to implementconcurrent verification before a step may be marked as completed, inwhich case an initial or signature sub-widget is added to a Step widget.10. The method of claim 3, wherein a widget associated with aConditional Action content block is configured, through at least one ofproperties or sub-widgets, to express one or more logical conditionsbased on dynamic input data, where all conditions are configured to bedynamically evaluated at job performance time and result in automaticnavigation to widgets associated with different sections and contentblocks of the procedure or instructions, with configuration defining oneor more of: multiple conditions linked through logical operators; eachcondition to define the next widget to navigate to when the conditionevaluates to true or a different next widget when it evaluates to false;declaring the widget as a Continuous Action, when the expressedcondition must follow the form: if at any time [condition], then[Action].
 11. The method of claim 3, further comprising one or more ofthe following: a widget associated with a Branch content block isconfigured to cause a transition to a procedure or instructions stepother than a next step in a current procedure or instructions or toanother digitized document containing other procedure or instructionswithout returning to current sections of the digitized document with thewidget associated with the Branch content block, at job performancetime; a widget associated with a Reference content block is configuredto cause a transition to perform actions from another document or atransition to a section in another set of written procedures orinstructions and then return to a current digitized document to completea task or activity, at job performance time; a widget associated with aComponent Verification content block is configured to provideinteraction means for integration to equipment sensors, barcodes, RFIDor similar technologies, or the ability to scan and analyze a componentor a tool's tag or ID for purposes of verifying that a correct equipmentor tool is being worked-on or used for a procedure or instructions athand; a widget associated with a Signature or Initials content block isconfigured to provide means for a user to electronically sign acorresponding section; a widget associated with a Figure content blockis configured to include a figure and may be configured to display thefigure or to collapse a section containing the figure such that it canbe expanded on demand by a user, at job performance time; a widgetassociated with a Table content block is configured to includesub-widgets designed for data-entry or dynamic data population of asection of the procedure or instructions at job performance time; awidget associated with a Hold Point content block is configured toenforce and support obtaining a signature of another qualified user forverification without requiring a verifier to be present, at jobperformance time to mark the step as complete; a widget associated witha Bulleted list content block is configured to provide interaction meansto perform a contained bulleted instructions in any order while keepingtrack with a current bullet, having marked bullets that were alreadyperformed and enforcing performance of all bullets; or a General Contentwidget, with configurable interaction capabilities, is associated with acontent block of the digitized document to capture the content of anassociated section and to provide relevant interaction throughconfiguration of its attributes.
 12. The method of claim 3, furthercomprising: a Repeat interaction widget is associated with a portion ofinstructions requiring repeating a series of steps based on dynamicallyevaluated conditions or a fixed number of iterations where said Repeatwidget automatically provides all required navigation, computation,branching and placekeeping interactions as well as a clear view of alliterations performed; a widget associated with a Step is configured tocontain nested sections and Steps and their corresponding widgets. 13.The method of claim 1, further comprising: programmatically reading thedigitized document into computer memory; programmatically analyzing thecontents of the digitized document; programmatically matching contentblocks with associated widgets; programmatically superimposing oroverlaying interaction widgets associated with sections and contentblocks of the digitized document; programmatically sequencing thewidgets; programmatically inferring other attributes; andprogrammatically saving the resulting structured metadata such that thestructured metadata can be displayed for further interaction design orvalidation.
 14. In a computing system designed for having one or morecomputers configured to implement a method to dynamically integrate dataor cause the dispatch of software functions, from external softwareapplications, in the context of select areas of the content blocks ofdigitized written procedure or instructions, an instructional manual ora fillable form, or within widgets containing said content blocks, amethod comprising: obtaining a digital version of a digitized document,the digitized document comprising a written procedure or instructions,instructional manual or fillable form; implicitly or through explicitconfiguration, associating each possible type of input or output data ofsoftware services to a typed interface widget; providing a means for theselection of a specific input or output of a software service availablewhere said software service may be used as a means to pass data or bringdata or dispatch a function to or from an external application;providing a means to superimpose or overlay the widget associated tosaid input or output on top of an area of a content block of saiddigitized document or within other widgets that are designed toencapsulate and contain such content block; providing a means to definethe order of appearance and activation of each widget relative to otherwidgets at job performance time; providing a means to configure theattributes of the widgets; providing a means to save the resultingwidget information, including at least one of: information about acontent block contained within each widget, a widgets position relativeto an associated content block or relative to a containing widget, or awidgets attributes, on a computer-readable storage medium, for executionor interpretation by one or more hardware processors to automaticallygather dynamic data for the inputs of the software service through itsassociated interface widget superimposed or overlaid on selected areasof a said content block, to automatically invoke the service, and topopulate widgets associated with the service outputs with output data ofthe service effectively over the selected areas of a said content blockof the digitized document at job performance time.
 15. The method ofclaim 14, further comprising: displaying the digitized document in agraphical user interface; providing a means to access graphical userinterface input or output definitions of a software service fordescribing the inputs or outputs of a service including data types,optional defaults, or restrictions for each input or output field;providing user interface elements to allow a user to select a serviceinput or output field and then automatically display a data widgetimplicitly, or explicitly through configuration, associated with saidinput or output field and allowing the user to place the widget on anarea of the display over a content block of the digitized document beingdisplayed, or over another widget containing said content block, andaccommodating the resizing and relocation of the widget by the user tocause the widget to either overlay or to superimpose over the selectedsaid content block or its containing widget, for capturing dynamic userinput data to be automatically mapped to the inputs of an associatedsoftware service, or for populating a specific area of said contentblock with dynamic application data from a service output, at jobperformance time, after the automatic execution of the associatedservice; when an interface widget needs to occupy space that is largerthan space provided by boundaries of a contained content type, modifyingthe digitized document to add the required space, extending said widgetto occupy the added space, readjusting coordinates of other widgets thatare in a location of the display below the extended widget to compensatefor the added space and displaying the modified version instead of theoriginal digitized document on the display; providing a user interfaceelement to allow the user to place an interface widget inside of anotherwidget, where the other widget may be used in order to contain orcapture the contents of a content block of the digitized document;adding to each interface widget metadata including coordinates of thewidget relative to the displayed digitized document such that saidwidget may be superimposed or overlaid on top of the digitized documentat job performance time; when an interface widget is contained inanother widget, adding to widget metadata the coordinates of theinterface widget relative to the containing widget, such that at jobperformance time, the containing widget and its captured content blockmay be displayed first and then the interface widget can be displayedrelative to the containing widget irrespective of the position of saidinterface widget relative to the digitized document; providing a userinterface element to allow the user to set the look & feel properties orchange the associated default type of the widget to a different widget;storing and referencing within an interface widget, user definedconstants with their default values; defining the order of appearance,in order to support computer-guided navigation of the widgets at jobperformance time, by providing means to sequence and link one widget toanother, either statically or based on dynamically evaluated expressedconditions and adding such sequencing information to the widget datastructures; saving, on a persistent computer data store each widget'slook & feel properties, other user attributes as well as similarmetadata of contained and containing widgets and other metadataassociated with the widgets, including: i) a copy of the displayeddigitized document together with the position of overlaid orsuperimposed widgets relative to said digitized document or relative toits containing widget, ii) content of content blocks contained by eachcontaining widget, in a structured format, for interpretation at jobperformance time.
 16. The method of claim 15, wherein one or more of thefollowing interface widget types is provided: a Data Interface widgetthat is not tied to the inputs or outputs of software services and isused to collect input data dynamically and without invoking a service atjob performance time and allow a user and the external application toreference dynamic data within other widgets in order to supportconditional navigation and to perform computations; an Equipmentinterface widget that integrates events and data from specific IoTenabled equipment or to cause such equipment to perform a specificaction at job performance time; a configurable Notification interfacewidget used to cause the automatic publication of a notification basedon the evaluation of dynamic data to a person performing a job or toother applications through automatic dispatch of a software serviceassociated to said widget.
 17. In a computing system designed for havingone or more computers configured to implement a method to performautomated software application integration, user-computer interactions,computer guided navigation or automation functionality for user orcomputerized machines performing a job, with said job relating tocontent blocks of a digitized document comprising at least one ofwritten procedure or instructions, an instructional manual or a fillableform, with said content blocks being encapsulated within sequenced andlinked widgets with specific interaction, integration and navigationbehavior, and based on information contained in structured metadata, amethod comprising: loading structured metadata in application memory ina structure appropriate for execution or interpretation by various typesof processors when a work associated with an encapsulated content blockof a corresponding digitized document is being performed by a person,group of people or one or more computerized machines; displaying thedigitized document on a device display; activating and performinginteractions, automations and integrations of a first widget, associatedwith a first encapsulated content block of the digitized document beingdisplayed, based on the structured metadata; activating and performinginteractions, automations and integrations of relevant sub-widgets;determining a next widget that needs to be activated based on static orconditional information in the metadata that depends on dynamic datagenerated or gathered during performance of content blocks associated towidgets at job performance time; performing interactions, automationsand integrations of the next widget; and repeating the acts ofdetermining a next widget performing interactions, automations andintegrations of the next widget until a last activated widget isreached.
 18. The method of claim 17, further comprising: whereindisplaying the digitized document on a device display comprises eitherdisplaying the digitized document in a graphical user interface, orwithout relying on the whole of the digitized document, displaying allcontent blocks associated with each widget on the device display bydetermining or assigning an appropriate location for a first widget,corresponding to a first content block and then determining locations ofall the other linked widgets relative to the first or a previous widgetor a containing widget and then displaying a content block containedwithin each widget inside a location determined for each widget;displaying or activating the first widget associated with the firstdisplayed content block by superimposing or overlaying it over theassociated content block based on properties expressed in the widgetmetadata, and then proceeding to the display of a next widget in asimilar fashion, after the interaction of the first widget is completed;repeating the process until all superimposed or overlaid widgets,required for a job performance are displayed or a last section of thedigitized document is reached and a last widget is displayed andassociated interactions performed; when displaying an interface widget,based on information in the widget metadata, either automaticallyinvoking all software services whose outputs are associated with thewidget and then populating or displaying the output data, resulting fromthe automatic invocation of the service, inside of the associatedinterface widget; or, prompting the user for input data, using anappropriate widget associated to a type of input and then transferringuser supplied inputs to inputs of the associated service and invokingthe service to perform a function on another application; and whendisplaying an interaction widget, and of its child widgets, performinguser-computer interaction implied by a specific type of the interfacewidget and displaying any contained interface widgets, and thenadvancing to a next relevant linked widget based on information in thestructured metadata and displaying a containing widget associated with anext section of procedure or instructions.
 19. The method of claim 17,wherein real-time state, data and the statistics associated with theperformance of a procedure or instructions based on the interactions andadvancement through widgets is automatically recorded, calculated,inferred and reported; wherein such statistics include one or more of:real-time reporting of a current section or step of procedure orinstructions, location of performance of a section based on GPScoordinates, start-time and completion time of each procedure orinstructions section or step, or analytics based on aggregatedstatistics across all job instances associated with a procedure orinstructions.
 20. The method of claim 18, wherein widgets associated tosome select sections of the procedure or instructions are superimposedor overlaid on top of the digitized document and the performance of someother select sections, including steps within select instructions, aretransferred to another device.